Likelihood ratio test (LRT) using difference of Objective Function Values (OFVs)
Non-nesting models:
Akaike Information Criterion (AIC)
Bayesian Information Criterion (BIC)
Model evaluations:
Parameter uncertainty (week 6)
Parametric: asymptotic standard errors ($COV, need to “covariance step successful”)
Non-parametric: Bootstrap or SIR
Model evaluations (week 7):
Diagnostic plots: PRED/IPRED vs DV, NPDE v TIME, etc.
Predictive checks.
Example model selection factors1
S: Successful convergence. S = Successful, E = Potential error during model run.
C: Successful covariance step. S = Successful, E = Error during covariance step
OFV: Objective function value.
dOFV: Difference of objective function value, relative to the reference model.
AIC: Akaike’s Information Criteria.
BIC: Bayesian Information Criteria.
Nesting
Concept: A model is nested within another model if the larger model reduces exactly to the smaller model by setting part of the larger model to its null value, without changing the interpretation of remaining parameters.
Base model constructions
One- vs. two-compartment model.
Add or remove an ETA on a parameter
Covariate modeling
Setting body weight effect (THETA(2)) to its null value (e.g., 0).
TVCL = THETA(1)*(WT/70)**THETA(2)
Setting sex effect (THETA(2)) to its null value (e.g., 0).
TVCL = THETA(1)*THETA(2)**SEX
Example of nesting models: one vs two-compartment models
If fixing \(Q=0\) (i.e., no distribution), then structurally, a two-compartment model (the larger model) reduces to a one-compartment model (the smaller model).
One-compartment model is nested within two-compartment model.
Example of nesting models: add or remove an ETA
Model 1: CL = THETA(1)
Model 2: CL = THETA(1) * EXP(ETA(1))
Fixing \(\omega^2_{(1,1)}\) (variance of ETA(1) distribution) as 0:
No between subject variability (i.e., zero variance)
Model 2 (the larger model) reduces to Model 1 (the smaller model).
Model 1 is nested within Model 2.
Degree of Freedom (DF)
The difference in the number of parameters between two nested models (competing vs. base models).
Base model: CL = THETA(1)
Competing model: CL = THETA(1) * EXP(ETA(1))
DF=1 when comparing two models.
Nesting model comparison: likelihood ratio test (LRT)
If two models are nested, then we can use likelihood ratio test (LRT) to test if the difference in model fittings is statistically significant.
In this case, goodness of model fitting is represented by the objective function value (OFV) of each model.
OFV calculation details: lecture in week 10
Assumptions:
The difference in the OFV between two nested models (dOFV) is approximately Chi-square distributed with degree of freedom (DF) equals to the number of additional parameters.
One vs two-compartment model: DF=2 (CL and V1 vs. CL, V1, Q, and V2).
Add one more ETA: DF=1
Nesting model comparison: LRT test criteria
For a single LRT, we often use \(\alpha=0.05\) (i.e., \(dOFV \geq 3.84\)).
Many publications did this.
However, in practice, we typically perform multiple LRT (add multiple parameters) while developing a popPK model.
Therefore, strictly speaking, \(dOFV \geq 3.84\) does not control enough for multiple testing.
\[
\frac{dC_p}{dt} = -K_{on} \times C_p \times R + K_{off} \times RC - K_{el} \times C_p\\
\frac{dR}{dt} = K_{syn}-K_{deg}\times R-K_{on} \times C_p \times R + K_{off} \times RC \\
\frac{dRC}{dt} = K_{on} \times C_p \times R - K_{off} \times RC
\] We cannot reduce one model to make it the same as the other via fixing part of the model to null value (i.e., non-nesting models).